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TECHNICAL INFORMATION 



No. TW0007 



(3) Brief description of tho Invention 



Refer to documentation submitted to JP Fortin during his March 12/98 visit. 



jb) What the problem sotved by the invention? 



Refer to documentation submitted to JP Fortin during his March 12/98 visit. 
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TECHNICAL INFORMATION (continued] 



No. 



TW0007 



\c\ What other solutions have been tried and what were their shortcomings? 



1 . We tried modifying The arbitration algorithm to continue granting the PCI bus to a card until it finished a transfer without 
getting a disconnect. The intention of dofng this was to maintain the byte order of the ATM cells. This didn't work because it 
resulted in the ATM cells being written to the bridge as two bursts instead of one when a disconnect occured. Even though 
the byte order of the cell is intact in the bridge, the bridge will transmit the cell as two seperate bursts, not one single burst. 
This allows the arbiter on the primary side of the bridge to grant the bus to another Initiator that is sending to the same 
target as the bridge between sending the first and second part of the cell from the bridge. This will prevent the cell from the 
bridge from arriving intact at the target. 

2. We tried adjusting the priority of the bridges on the primary side to keep the bridge buffers empty. This only works if there 
are 2 or less initiators on the secondary side and only one secondary bus. The bridge we were using has a buffer large 
enough to hold only 2 1/2 ATM cells. If 3 or more initiators were to burst cells to the bridge back to back, the bridge may 
not be able to write out the first cell before the last cell arrived which would resurt in a disconnect and the breaking up of the 
ATM cell. 

3. We investigated redesigning our target cards to provide a seperate buffer for each initiator* The cost and board area Of this 
approach were not attractive and the idea was dropped. 

4. We talked about having very large buffers on the bridge. This would require customization of the off-the-shelf bridge chip 
which is not a short term option. This is probably the best long term solution but it did not meet our time requirements. 



(dl What afe the specific elements or steps that soared the problem. Pte3$e provide high level details. 



ATM cells are kept intact while crossing the bridge by preventing disconnects from happening in the middle of the burst of an 
ATM ceil to the bridge. This is done by ensuring there is enough room in the buffer to receive a complete ATM cell- If there 
isn't enough room the arbiter inhibits grants to all initiators on the bus. 



lot What Is the commercial value of ttie invention to Nortel and Nortel's major competitors? (see 3u^eiine$)_ 



This method allows the following: use of bridges to expand slot count in a CPCI chassis, use of a PCI bus for both control 
and data transfer, use of a single receive buffer on target cards. 

The Compact PCI spec, limits the number of slots per PCI bus to 8 including the processor board. This leaves 7 useable 
expansion slots. With 2 bridges in a chassis we can add an additional 2 PCI busses to allow 19 expansion slots in a chassis. 
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What other solutions have been tried? 

1 . We tried modifying the arbitration algorithm to continue granting the PCI bus to a card until it finished a 
transfer without getting a disconnect. The intention of doing this was to maintain the byte order of the 
ATM cells. This didnt work because it resulted in the ATM cells being written to the bridge as two bursts 
instead of one when a disconnect occured. Even though the byte order of the cell is intact in the bridge, the 
bridge will transmit the cell as two seperate bursts, not one single burst. This allows the arbiter on the 
primary side of the bridge to grant the bus to another initiator that is sending to the sametarget asThe 
bridge between sending the first and second part of the cell from the bridge. This will prevent the cell from 
the bridge from arriving intact at the target 

2. We tried adjusting the priority of the bridges on the primary side to keep the bridge buffers empty. This 
only works if there are 2 or less initiators on the secondary side and only one secondary bus. The bridge 
we were using has a buffer large enough to hold only 2 1/2 ATM cells. If 3 or more initiators were to burst 
cells to the bridge back to back, the bridge may not be able to write out the first ceil before the last cell 
arrived which would result in a disconnect and the breaking up of the ATM cell. 

3. We investigated redesigning our target cards to provide a seperate*"buffer for each initiator. The cost and 
board area of this approach were not attractive and the idea was dropped. 

4. We talked about having very large buffers on the bridge. This would require customization of the off- 
the-shelf bridge chip which is not a short term option. This is probably the best Jong term solution but it did 
not meet our time requirements. 

What are the specific elements or steps that solved the problem and how do they do it? 

ATM cells are kept intact while crossing the bridge by preventing disconnects from happening in the 
middle of the burst of an ATM cell to the bridge. This is done by ensuring there is enough room in the 
buffer to receive a complete ATM cell. If there isnt enough room the arbiter inhibits grants to all initiators 
on the bus. 

What is the commercial value of the invention to Nortel and Nortel's major competitors? 

This method allows the following; use of bridges to expand slot count in a CPC1 chassis, use of a ?CI bus 
for both control and data transfer, use of a single receive buffer on target cards. 

The Compact PCI spec, limits the number of slots per PCI bus to 8 including the processor board. This 
leaves 7 useable expansion slots. With 2 bridges in a chassis we can add an additional 2 PCI busses to 
allow 19 expansion slots in a chassis. 
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SPTidin fl ATM Cells Across PCI to PCI Bridges With Small Buffers From The 

Secondary to Primary Side 

Prob.temj 

Nortel -BWA is using the PCI bus as a medium to transfer ATM cells between cards in a chassis based 
Compact PCI product. Since the Compact PCI bus is limited to 8 loads per bus segment Pd to PCI bridges 
arc used to extend the number of slots that can be supported in a single chassis. In order to maximize the 
bandwidth of the PCI bus, all cards in the system transfer their outgoing ATM cells by performing a write 
burst of a 14 Dword ATM cell to a preprogramed target address. Using writes to transfer data allows 
write posting across PCI to FCI bridges which maximizes bandwidth utilization. If the target address is on 
a different bus segment than the initiator, the burst will be write posted into the bridge's buffer. If an 
initiator is bursting an ATM cell across the bridge and the bridge buffer fills up, then the bridge will do a 
target disconnect even if the ATM cell has not been completely transferred. The arbiter will grant the bus 
to the next initiator which will write a complete ATM cell over top of the partial ATM cell that is already in 
the buffer. The initiator that was disconnected will finish sending its ATM eel! when it is next granted the 
bus. If all of the cells crossing the bridge were to the same target address, the cells will be corrupted when 
they arrive at the target because of the disconnect ihat happened while crossing the bridge. 

Certain cards in the system, such as modems and cell multiplexers, will have multiple cards writing to a 
single FIFO target location on that card- In order for these cards to operate properly the 14 DWord bursts 
must be written into the FIFO as complete cells and not as partial cells, i.e. the ATM cells must arrive in 
their entirety and not be broken up. 

A summary of the problem is that target disconnects must be prevented when data from multiple initiators 
intended for a single target address crosses a PCI to PCI bridge to ensure that ATM cells arrive intact; 

Solution: 

The CPCI architecture is as shown in figure I . Arbitration on Bus 1 is controlled by the Host to PCI bridge 
thai physically resides on the host CPU. Arbitration on Busses 2 and 3 is controlled by the PCI to PCI 
bridge connected to them. The following assumptions are made when using this architecture: 

- all cards that have a target address on them that will be simultaneously written to by multiple 
initiators must be on the Bus 1. 
' - all cards that have a target address on them that will be simultaneously written to by multiple 
initiators must never cause a "target disconnect with data" when being written to. 'Target 
disconnect without data" is acceptable because the cycle is terminated before any data is sent. 

Based on these assumptions, disconnects must only be prevented when data is being written from busses 2 
or 3 to bus 1. This is because disconnects are Only a problem when multiple initiators are writing to a 
single target which by definition is on bus 1 . 

Disconnects on busses 2 and 3 are avoided by preventing any of the cards on the secondary bus from being 
granted the bus when there isn't enough room in the bridge buffer for a complete ATM cell. A counter 
exists on the PC to PCI bridge module that is incremented when a card on its secondary side starts a write 
transaction to the bridge. The write may be a single word or a multi DWord burst and will be referred to a 
data unit The counter is decremented when the bridge successfully completes a write transaction on its 
primary side. The counter is not decremented if the bridge gets a target disconnect since this indicates that 
the data unit was not completely sent. If the counter value is equal to the maximum number of data units 
that the secondary to primary buffer on the bridge can hold, then the arbitration on the secondary bus is 
parked and the bus is granted to the bridge whether it is requesu'ng or not Arbitration is not affected when 
the count value is less than this maximum value. The size of the data unit is assumed to the maximum size 
of a burst which is 14 DWords. 
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Figure 1 .- Nortel -BW A Compact PCI Architecture 

Counting da*, units requires only a stnall counter and thus minimal hardware resources. The '«^ io » to f 
; me*od is that it^Lumes that all data units are the same size, which is genially the case If Uw ^> , of 
he data units varies, the buffer is inefficiently utilized. If the size of the data units vanes*. 
lis method is to count DWords instead of data units and to disable the grants to the secondary bus if there 
is not enough room for the maximum sue of a burst. This requires more hardware to implement. 
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TECHNICAL INFORMATION No. TWOOO 



(d) Brief d&scnptjon^f the Invention 



Refer to documentation submrtted to JP Fortin during his March 1 2/98 visit. 




|(bl What Is the problem soiled by the Invention? 



Refer to documentation submitted to JP Fortin during his March 12/98 visit. 



i 
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TECHNICAL INFORMATION (continued) 



No. 



TW0007 



(cl What other solutions have fcocn tried end what were their shortcomings 



1 w„ tried modifvina the arbitration algorithm to continue granting the PCI bus to a card until it finished a transfer without 
e*ng Hl^nS^t intention of doing this we, to maintain the byte order of the ATM cell,. This ™™£2XZ? 

?eVu ted in the ATM cells being written to the bridge as two bursts instead of one when a disconnect occured. Even though 
he byte oX of the cell is inTact m the bridge, the bridge wi.l transmit the ce.l as two seperate bursts. n«rt W e s,ng^ burst. 
This allows the arbiter on the primary side of the bridge tp grant the bus to another initiator that is sending to the same 
targeras the bridge between sending the first and second port of the cell from the bridge. This wdl prevent the cell from the 
bridge from arriving intact at the target. 

2 We tried adjusting the priority of the bridges on the primary side to keep the bridge buffers empty. This only works if thl 
are 2 or less initiators on the secondary side and only one secondary bus. The bridge we were using has a buffer large 
enough to hold only 2 1/2 ATM cells. If 3 or more initiators were to burst cells to the bridge back to back, the bndge may 
not be able to write out the first ceit before the last cell arrived which would result in a disconnect and the breaking up of | 
ATM cell. 

3. We investigated redesigning our target cards to provide a seperate buffer for each initiator. The cost and board area of this 
approach were not attractive and the idea was dropped. 

4 we talked about having very large buffers on the bridge. This would require customization of the off-the-shelf bridge chip 
which is not a short term option. This is probably the best long term solution but rt did not meet our time requirements. 



.JJ^ lF i l JJJ...J. I f]J.ir.k,^ 



(d) What ore 



ATM cells are kept intact while crossing the bridge by preventing disconnects from happening in the middle * *>• of an 
ATM cell to the bridge. This is done by ensuring there is enough room in the buffer to receive a complete ATM cell. If there 
isn't enough room the arbiter inhibits grants to all initiators on the bus. 



(c| What Is the commercial vatue of the Indention to Nortel qntf Worry's major competit ors? (see guidelines) 



This method allows the following: use of bridges to expand slot count in a CPCI chassis, use of a PCI bus for both control 
and data transfer, use of a single receive buffer on target cards. 

The Compact PCI spec, limits the number of slots per PCI bus to 8 including the processor board. This leaves 7 useable 
expansion slots. With 2 bridges in a chassis we can add an additional 2 PCI busses to allow 19 expansion slots in a chassis. 
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What other solutions have been tried? 

1 . We tried modifying the arbitration algorithm to continue granting the PCI bus to a card until it finished a 
transfer without getting a disconnect The intention of doing this was to maintain the byte order of the 
ATM cells. This didnt work because it resulted in the ATM cells being written to the bndge as two bursts 
instead of one when a disconnect occured. Even though the byte order of the cell is intact in the bridge, the 
bridge will transmit the cell as two seperate bursts, not one single burst. This allows the arbiter on the 
primary side of the bridge to grant the bus to another initiator that is sending to the sametargetas-the 
bridge between sending the first and second part of the ceil from the bridge. This will prevent the cell from 
the bridge from arriving intact at the target 

2. We tried adjusting the priority of the bridges on the primary side to keep the bridge buffers empty. This 
only works if there are 2 or less initiators on the secondary side and only one secondary bus. TTie bridge 
we were using has a buffer large enough to hold only 2 1/2 ATM cells. If 3 or more initiators were to burst 
cells to the bridge back to back, the bridge may not be able to write out the first cell before the last cell 
arrived which would result in a disconnect and the breaking up of the ATM cell. 

3. We investigated redesigning our target cards to provide a seperate* buffer for each initiator. The cost and 
board area of this approach were not attractive and the idea was dropped. 

4. We talked about having very large buffers on the bridge. This would require customization of the off- 
the-shelf bridge chip which is not a short term option. This is probably the best long term solution but it did 
not meet our time requirements. 

What are the specific elements or steps that solved the problem and how do they do it? 

ATM cells are kept intact while crossing the bridge by preventing disconnects from happening in the 
middle of the burst of an ATM cell to the bridge. This is done by ensuring there is enough room in the 
buffer to receive a complete ATM cell. If there isn't enough room the arbiter inhibits grants to all initiators 



What is the commercial value of the invention to Nortel and Nortel's major competitors? 

This method allows the following: use of bridges to expand slot count in a CPCI chassis, use of a PCI bus 
for both control and data transfer, use of a single receive bufTer on target cards. 

The Compact PCI spec, limits the number of slots per PCI bus to S including the processor board. This 
leaves 7 useable expansion slots. With 2 bridges in a chassis we can add an additional 2 PCI busses to 
allow 1 9 expansion slots in a chassis. 



on the bus. 



Received from < 604 682 0274 > at 3/18/03 4:35:43 PM [Eastern Standard Time] 



MAR 18 2003 13:29 FR 604 6j 



.0274 



604 682 0274 TO 



337465913 



Sending ATM Cells Across PCI to PCI Bridges With Small Buffers From The 

Secondary to Primary Side 



Problem: 

Nortel -BWA is using the PCI bus as a medium to transfer ATM cells between cards in a chassis based 
Compact PCI product. Since the Compact PCI bus is limited to 8 loads per bus segment PCI to PCI bridges 
are used to extend the number of slots that can be supported in a single chassis, In order to maximize the 
bandwidth of the PCI bus, all cards in the system transfer their outgoing ATM cells by performing a write 
burst of a 14 Dword ATM cell to a preprogrammed target address. Using writes to transfer data allows 
write posting across PCI to PCI bridges which maximizes bandwidth utilisation. If the target address is on 
a different bus segment than the initiator, the burst will be write posted into the bridge's buffer. If an 
initiator is bursting an ATM cell across the bridge and the bridge buffer fills up, then the bridge will do a 
target disconneel even if the ATM cell has not been completely transferred- The arbiter will grant the bus 
to the next initiator which will write a complete ATM cell over top of the partial ATM cell thai is already in 
the buffer. The initiator that was disconnected will finish sending its ATM cell when it is next granted the 
bus. If all of the cells crossing the bridge were to the same target address, the cells will be corrupted when 
they arrive at the target because of the disconnect that happened while crossing the bridge- 
Certain cards in the system, such as modems and cell multiplexers, will have multiple cards writing.to a 
single FIFO target location on that card. In order for these cards to operate properly the 14 DWord bursts 
must be written into the FIFO as complete cells and not as partial cells, i.e. the ATM cells must arrive in 
their entirety and not be broken up. 

A summary of the problem is that target disconnects must be prevented when data from multiple initiators 
intended for a single target address crosses a PCI to PCI bridge to ensure that ATM cells arrive intact; 

golution; 

The CPCI architecture is as shown in figure 1. Arbitration on Bus 1 is controlled by the Host to PQ bridge 
that physically resides on the host CPU. Arbitration on Busses 2 and 3 is controlled by the PCI to PCI 
bridge connected to them. The following assumptions are made when using this architecture: 

- all cards that have a target address on them that will be simultaneously written to by multiple 
initiators must be on the Bus 1 . 
" - all cards that have a target address on them that will be simultaneously written to by multiple 
initiators must never cause a "target disconnect with data" when being written to. 'Target 
disconnect without data" is acceptable because the cycle is terminated before any data is sent. 

Based On these assumptions, disconnects must only be prevented when data is being written from busses 2 
or 3 to bus 1. This is because disconnects are only a problem when multiple initiators are writing to a 
single target which by definition i$ on bus 1 . 

Disconnects on busses 2 and 3 are avoided by preventing any of the cards on the secondary bus from being 
granted the bus when there isn't enough room in the bridge buffer for a complete ATM cell. A counter 
exists on the PCI to PCI bridge module thai is incremented when a card on its secondary side starts a write 
transaction to the bridge. The write may be a single word or a multi DWord burst and will be referred to a 
data unit. The counter is decremented when the bridge successfully completes a write transaction on its 
primary side. The counter is not decremented if the bridge gets a target disconnect since this indicates that 
the data unit was not completely sent. If the counter value is equal to the maximum number of data units 
that the secondary to primary buffer On the bridge can hold, then the arbitration on the secondary bus is 
parked and the bus is granted to the bridge whether it is requesting or not. Arbitration is not affected when 
the count value is less than this maximum value. The size of the data unit is assumed to the maximum size 
of a burst which is 14 DWords. 



Received from < 604 682 0274 > at 3118/03 4:35:43 PM [Eastern Standard Time] 



MAR 18 2003 13:30 FR 604 6*^0274 604 682 0274 TO ^fc>374659 1 3 p 



Host Bus 




Figure 1 - Nortel - BWA Compact PCI Architecture 

Counting data unit* requires only a small counter and thus minimal hardware resource* The Hrnitatior . u> 
u° 3,^ ie ih»t it a « umeS ; that all data units are the same size, which is generally the case. If the siae oi 

J noTeS ^5>mX the maximum size of a burst. This retires more hardware to .mplement. 
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